An Overview of a Compiler for Scalable Parallel Machines

This paper presents an overview of a parallelizing compiler to automatically generate eecient code for large-scale parallel architec-tures from sequential input programs. This research focuses on loop-level parallelism in dense matrix computations. We illustrate the basic techniques the compiler uses by describing the entire compilation process for a simple example. Our compiler is organized into three major phases: analyzing array references , allocating the computation and data to the processors to optimize parallelism and locality, and generating code. An optimizing compiler for scalable parallel machines requires more sophisticated program analysis than the traditional data dependence analysis. Our compiler uses a precise data-ow analysis technique to identify the producer of the value read by each instance of a read access. In order to allocate the computation and data to the processors, the compiler rst transforms the program to expose loop-level parallelism in the computation. It then nds a decomposition of the computation and data such that parallelism is exploited and the communication overhead is minimized. The compiler will trade oo extra degrees of parallelism to reduce or eliminate communication. Finally, the compiler generates code to manage the multiple address spaces and to communicate data across processors.